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Attorney Docket No. 401 175/SAKAI 
IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re Application of: 
HIROSHI KAGE 
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For: METHOD AND DEVICE 

FOR GENERATING A 
PERSONS PORTRAIT, 
METHOD AND DEVICE 
FOR COMMUNICA- 
TIONS, AND COMPUTER 
PRODUCT 



SPECIFICATION, CLAIMS AND 
ABSTRACT AS PRELIMINARILY AMENDED 

Amendments to the paragraph beginning at page 1, line 6: 

The present invention relates to a technology for generating an image of a person's 
portrait in which, from a two-dimensional image containing a person's face picked up by 
an image sensor,4he a feature of the subject person is emphasized to generate the person's 
portrait or generate a deformed image of the person's portrait. This invention also relates 
to a method and a device for communication, and a computer readable recording medium 
which stores a computer program which 4 when executed on a computer realizes A the 
methods method according to the present invention. 

Amendments to the paragraph beginning at page 1, line 18: 

In recent years, it has become possible to send/receive electronic=»*ails mail not 
only on a personal computer but also on a portable terminal. Furthermore, not only 
documents, but also a sender's portrait have been widely attached to the electronic mailo 
mail . When creating such a portrait by using a conventional method, it is necessary for 
the user to manually make selections from prepared parts of several tens or more kinds, 
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such as face outline parts, face parts and hairstyles, and combine them all together. In this 
case, a large number of combinations are available, and A as a result, a great amount of 
workload is imposed on the user in an attempt to create a desired person's portrait. For 
this reason, there has been a strong demand for a device which can easily and 
automatically generate a person's portrait using an image of that person acquired by a 
camera. In order to create a person's portrait, such a device needs to automatically carry 
out a sequence of processes of extracting the face portion from the image, detecting eyes, 
mouth, etc., and replacing the desired parts with already prepared desired templates. 

Amendments to the paragraph beginning at page 2, line 23: 

(1) the face color is influenced by illumination conditions, resulting in a difficulty 
in detecting the face area= in some cases; 

Amendments to existing claims: 

1 . (Amended) A person's portrait generation device comprising: 

an image input section which picks up=a two-dimensional imago images 
containing a person's face=fey A using an image sensor; 

a head area extracting section which extracts a head area from a differential image 
of a plurality of the two-dimensional images picked up by the image input section; 

a feature detection section which detects position of characteristic features of the 
face within the extracted head area extracted : 

a face outline determining section which determines a border between a face 
outline and a background within the head area; and 

an image processing section which generates a person's portrait in which the 
charactoriotico in characteristic features of th e poroon'o face4s are emphasized based upon 
the two-dimensional imagery using data acquired4i% by the head area extracting section, 
the feature detection section^ and the face outline determining section. 

2. (Amended) The person's portrait generation device according to claim 1, 
wherein^ in the head area extracting section,-^© an outline of the head area is obtained by 
combining a right-side profile and a lift oido left-side profile of the differential image. 
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4. (Amended) The person's portrait generation device according to claim 2, 
wherein the head area extracting section provides the head area obtained by a hc£id 
rectangular shape in which a left side, a right side i and an upper side are determined by 
the outline of the head area and a lower side is determined by a shape ratio constant of=a 
general an average person's head. 

5. (Amended) A person's portrait generation device comprising: 

an image input section which picks up a two-dimensional image containing a 
person's face=by A using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 

a feature detection section which detects position of characteristic features of the 
face by dividing the head area thus extracted into fac e part parts areas for respective face 
parts and finding projection data of binarized images for the respective face parts areas; 

a face outline determining section which determines a border between a face 
outline and a background within the head area; and 

an image processing section which generates a person's portrait in which the 
charactcriotico in features of th e p croon 'o face=4s are emphasized based upon the two- 
dimensional image=fey using data acquired=k* by the head area extracting section, the 
feature detection section 4 and the face outline determining section. 

6. (Amended) The person's portrait generation device according to claim 5, 
wherein the face part s-aFGa-4s areas are determined by relative positional information of 
the face parts with respect to the head area preliminarily found. 

8. (Amended) The person's portrait generation device according to claim 5, 
wherein the position of characteristic features of the face is detected-tey based on a 
position of a maximum value of the projection data. 

9. (Amended) A person's portrait generation device comprising: 
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an image input section which picks up a two-dimensional image containing a 
person's face=%y using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 

a feature detection section which detects position of characteristic features within 
th e extracted head area extracted : 

a face outline determining section which determines a border between a face 
outline and a background within the head area by specifying a skin color area of the two- 
dimensional image; and 

an image processing section which generates a person's portrait in which the 
charactcriotico in characteristic features of th e pcroon'o face4s are emphasized based upon 
the two-dimensional imagery using data acquired=4s by the head area extracting section, 
the feature detection section,, and the face outline determining section. 

10. (Amended) The person's portrait generation device according to claim 9, 
wherein the face outline determining section determines=the outline of4b© a jaw. 

11 . (Amended) The person's portrait generation device according to claim 9, 
wherein, in the face outline determining section, the skin color area is determined by 
converting an inputted RGB value to an HSV value. 

12. (Amended) The person's portrait generation device according to claim 9, 
wherein, in determining the skin color area in the face outline determining section, with 
respect to an average colo r in the vicinity of near the characteristic features of the face, 
areas having similar colors are used as face area candidates. 

13. (Amended) A person's portrait generation device comprising: 

an image input section which picks up a two-dimensional image containing a 
person's face=^y using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 
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a feature detection section which detects position of characteristic features of the 
face for each of=4he face parts within the head area that has been extracted; 

a face outline determining section which determines a border between a face 
outline and a backgrouncLwithin the head area; and 

an image processing section which generates a person's portrait in which the 
characteristics in characteristic features of th e person's face4s are emphasized by 
changing the two-dimensional image with respect to each of=4fee face parts. 

14. (Amended) The person's portrait generation device according to claim 13, 
wherein the image processing section expresses a person's emotions by changing=fte 
shape of a partial image at each of an eye portion, a noioo nose portion A and a mouth 
portion that are the characteristic features of the face. 

15. (Amended) The person's portrait generation device according to claim 13, 
wherein the image processing section replaces partial images at an eye portion, a noise 
nose portion^ and a mouth portion that are characteristic features of the face. 

16. (Amended) A communication terminal comprising: 

an image input section which picks up=a two-dimensional image images 
containing a person's face-fey using an image sensor; 

a head area extracting section which extracts a head area from a differential image 
of a plurality of the two-dimensional images picked up by the image input section; 

a feature detection section which detects position of characteristic features of the 
face within th e extracted head area extracted : 

a face outline determining section which determines a border between a face 
outline and a background within the head area; and 

an image processing section which generates a person's portrait in which the 
charaotoricitico in characteristic features of th e poroon'o face4s are emphasized based upon 
the two-dimensional image=%y E A using data acquired=» by the head area extracting section, 
the feature detection section , and the face outline determining section, wherein the 
resulting person's portrait is transmitted and received through a communication unit. 
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17. (Amended) The communication terminal according to claim 16, wherein in 
the head area extracting section,=#*# an outline of the head area is obtained by combining 
a right-side profile and a lift side left-side profile of the differential image. 

19. (Amended) The communication terminal according to claim 17, wherein the 
head area extracting section provides the head area obtained by a head rectangular shape 
in which a left side, a right side x and an upper side are determined by the outline of the 
head area and a lower side is determined by a shape ratio constant o f a general an average 
person's head. 

20 (Amended) A communication terminal comprising: 

an image input section which picks up a two-dimensional image containing a 
person's face=fey using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 

a feature detection section which detects position of characteristic features of the 
face by dividing the head area=te*s extracted into fac e part parts areas for respective face 
parts and finding projection data of binarized images for the respective face parts areas; 

a face outline determining section which determines a border between a face 
outline and a background within the head area; and 

an image processing section which generates a person's portrait in which the 
characteristics in characteristic features of th e poroon'o face4s are emphasized based upon 
the two-dimensional imagery using data acquired4f% by the head area extracting section, 
the feature detection section^ and the face outline determining section, wherein the 
rcoulting person's portrait is transmitted and received through a communication unit. 

21. (Amended) The communication terminal according to claim 20, wherein the 
face part s area io areas are determined by relative positional information of the face parts 
with respect to the head area preliminarily found. 



A6 



In re Appln. of I^Bhi Kage 
Application No. ^Renown 



23. (Amended) The communication terminal according to claim 20, wherein the 
position of characteristic features of the face is detected-fey based on a position of a 
maximum value of the projection data. 

24. (Amended) A communication terminal comprising: 

an image input section which picks up a two-dimensional image containing a 
person's face-fey using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 

a feature detection section which detects position of characteristic features within 
th e extracted head area extracted : 

a face outline determining section which determines a border between a face 
outline and a background within the head area by specifying a skin color area of the two- 
dimensional image; and 

an image processing section which generates a person's portrait in which the 
charactcriGtico in characteristic features of th e person's face4s are emphasized based upon 
the two-dimensional image=fey using data acquired^ by the head area extracting section, 
the feature detection section,, and the face outline determining section, wherein the 
resulting p erson's portrait is transmitted and received through a communication unit. 

25. (Amended) The communication terminal according to claim 24, wherein the 
face outline determining section determines=4fee outline of4fe© a jaw. 

26. (Amended) The communication terminal according to claim 24, wherein A in 
the face outline determining section, the skin color area is determined by converting an 
inputted RGB value to an HSV value. 

27. (Amended) The communication terminal according to claim 24, wherein, in 
determining the skin color area in the face outline determining section, with respect to an 
average color in the vicinity of near the characteristic features of the face, areas having 
similar colors are used as face area candidates. 
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28. (Amended) A communication terminal comprising: 

an image input section which picks up a two-dimensional image containing a 
person's face^y using an image sensor; 

a head area extracting section which extracts a head area from the image picked up 
by the image input section; 

a feature detection section which detects position of characteristic features of the 
face for each of=Ae face parts within the head area that has been extracted; 

a face outline determining section which determines a border between a face 
outline and a background within the head area; and 

an image processing section which generates a person's portrait in which the 
characteristics in characteristic features of th e p croon 'o face4s are emphasized by 
changing the two-dimensional image with respect to each of4he face parts, wherein the 
resulting person's portrait is transmitted and received through a communication unit. 

29. (Amended) The communication terminal according to claim 28, wherein the 
image processing section expresses a person's emotions by changing=4fe§ shape of a 
partial image at each of an eye portion, a noioo nose portion, and a mouth portion that are 
the characteristic features of the face. 

30. (Amended) The communication terminal according to claim 28, wherein the 
image processing section replaces partial images at an eye portion, a noioo nose portion 4 
and a mouth portion that are characteristic features of the face. 

3 1 . (Amended) A person's portrait generation method comprising: 

an imago input otcp of p icking up=a two-dimensional imago images containing a 
person's face=¥y using an image sensor; 

a head area extracting stop of extracting a head area from a differential image of a 
plurality of the two-dimensional images picked up in the image input otop ; 

a feature detection atop of detecting position of characteristic features within the 
extracted head area extracted ; 



A8 



Appl: 



In re 



Appln. of: 
ication No 




|hi Kage 
nown 



a face outline determining step of determining a border between a face outline and 
a background within the head area; and 

an image proceeding step of creating a person's portrait in which the 
charactcriotico in characteristic features of th e person' o face are emphasized based upon 
the two-dimensional image^^ using output data obtained in extracting the head area 
extracting otcp , the feature detection otcp detecting position of characteristic features, and 
the face outline determining=st#p the border . 

32. (Amended) The person's portrait generation method according to claim 31, 
wherein, in extracting the head area extracting otcp ,=4he an outline of the head area is 
obtained by combining a right-side profile and a lift aide left-side profile of the 
differential image. 

33. (Amended) The person's portrait generation method according to claim 32, 
wherein the outline of the head area is obtained by eliminating noise components of the 
right-side profile and left-side profil e through a by filtering otcp . 

34. (Amended) The person's portrait generation method according to claim 32, 
wherein extracting the head area extracting otcp provides the head area that io obtained by 
in a head rectangular shape in which a left side, a right side A and an upper side are 
determined by the outline of the head area^ and a lower side is determined by a shape ratio 
constant o f a general an average person's head. 

35. (Amended) A person's portrait generation method comprising: 

an imago input otcp of picking up a two-dimensional image containing a person's 
face=fey using an image sensor; 

a head area extracting step ef=extracting a head area from the image picked up4s 
the image input ste p; 

a feature detection atop of detecting position of characteristic features of the face 
by dividing the head area thuo extracted into face=paft parts areas for respective face parts 
and finding projection data of binarized images for the respective face parts areas; 
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a face outline determining ste p of determining a border between a face outline and 
a background within the head area; and 

an imago procoooing otcp of creating a person's portrait in which the 
charactoriotico in characteristic features of th e pcroon'o face4s are emphasized based upon 
the two-dimensional image=ky A using data acquired in extracting the head area extraction 
slep, the feature detection otcp detecting position of the characteristic features and-the 
face outline determining=s*ep a border . 

36. (Amended) The person's portrait generation method according to claim 35, 
wherein the face part s area is areas are determined by relative positional information of 
the face parts with respect to the head area preliminarily found. 

38. (Amended) The person's portrait generation method according to claim 35, 
wherein the position of characteristic features of the face is detected=fe^ based on a 
position of a maximum value of the projection data. 

39. (Amended) A person' s portrait generation method comprising: 

an imago input otcp of p icking up a two-dimensional image containing a person's 
face=fey using an image sensor; 

a head area extracting otcp of extracting a head area from the image picked up4#* 
the image input otcp ; 

a feature detection otcp of detecting position of characteristic features within the 
extracted head area extracted ; 

a face outline determining otcp of determining a border between a face outline and 
a background within the head area by specifying a skin color area of the two-dimensional 
image; and 

an image procoooing otcp of creating a person's portrait in which the 
charactoriotico in characteristic features of the person' o face4s are emphasized based upon 
the two-dimensional image=^y A using data acquired in extracting the head area extraction 
sl#p, the feature detection st ep detecting position of the characteristic features, and=Ae 
face outline determining=st@p a border . 
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40. (Amended) The person's portrait generation method according to claim 39, 
wherein^ i n the face outline determining otop determining a border , the outline of4be a 
jaw is determined. 

41. (Amended) The person's portrait generation method according to claim 39, 
wherein A in the face outline determining otcp determining the border , the skin color area is 
determined by converting an inputted RGB value to an HSV value. 

42. (Amended) The person's portrait generation method according to claim 39, 
wherein i =A^ in determining method in the face outline a border, determining-step, with 
respect to an average color in the vicinity of the characteristic features of the face, areas 
having similar colors arc uacd as face area candidates. 

43. (Amended) A person's portrait generation method comprising: 

an imago input otop of p icking up a two-dimensional image containing a person's 
face=fey using an image sensor; 

a head area extracting otcp of extracting a head area from the image picked up4a 
the image input otop ; 

a feature detection otcp of detecting position of characteristic features of the face 
for each of the face parts within the head area that has been extracted; 

a face outline determining otop of determining a border between a face outline and 
a background within the head area; and 

an imago proooooing otop of creating a person's portrait in which the 
charactoriotico in characteristic features of th e pcroon'o face4s are emphasized by 
changing the two-dimensional image with respect to each of the face parts. 

44. (Amended) The person's portrait generation method according to claim 43, 
wherein tho imago proooooing otop oxprcoooo a pcroon'o emotions by a changing=tke shape 
of a partial image at each of an eye portion, a noioo nose portion^ and a mouth portion that 
are the characteristic features of the face in treating the person ! s portrait to express the 
person's emotions . 
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45. (Amended) The person's portrait generation method according to claim 43, 
wherein the imago proocooinc otop roplaooo . in treating the person's portrait, replacing 
partial images at an eye portion, a noioc nose portion and a mouth portion that are the 
characteristic features of the face. 

46. (Amended) A communication method using a communication terminal4fcal 
comprises comprising : 

an image input otop of picking up=a two-dimensional imago images containing a 
person's face^y using an image sensor; 

a head area extracting otop of e xtracting a head area from a differential image of a 
plurality of the two-dimensional images picked up in the image input otcp ; 

a feature detection otop of detecting position of characteristic features within the 
extracted head area extracted ; 

a face outline determining otcp of determining a border between a face outline and 
a background within the head area; 

an image proceeding otop of creating a person's portrait in which the 
charactcriotico in characteristic features of th e pcroon ' o face4s are emphasized based upon 
the two-dimensional image^y^ using output data obtained in extracting the head area 
extracting otcp , the feature detection otcp detecting position of characteristic features, and 
determining the border ; and 

a otcp in which, w hen a communication is received, the person's portrait thus 
created is displayed so as to inform4ke a user of=#*@ receipt of the communication. 

47. (Amended) A communication method using a communication terminal=feat 
comprioco comprising : 

an image input otcp of p icking up a two-dimensional image containing a person's 
face=fey using an image sensor; 

a head area extracting otcp of extracting a head area from the image picked up4s 
the image input otcp ; 
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a feature detection step of detecting position of characteristic features of the face 
by dividing the head area t huo extracted into face=ps#l parts areas for respective face parts 
and finding projection data of binarized images for the respective face parts areas; 

a face outline determining otop of determining a border between a face outline and 
a background within the head area; 

an image pro c cooing otcp of creating a person's portrait in which the 
charactcriotico in characteristic features of th e person* o face4s are emphasized based upon 
the two-dimensional image=fey A using data acquired in extracting the head area extraction 
slep, the feature detection otop detecting position of the characteristic features, and4fee 
face outline determining otcp a border ; and 

a otcp in which, when a communication is received, the person's portrait thus 
created is displayed so as to inform=A# a user of=A© receipt of the communication. 

48. (Amended) A communication method using a communication terminal that 
comprises: 

an image input otop of picking up a two-dimensional image containing a person's 
face=^y using an image sensor; 

a head area extracting otcp of extracting a head area from the image picked up4t* 
the image input otcp ; 

a feature detection otop of detecting position of characteristic features within the 
extracted head area extracted ; 

a face outline determining otop of determining a border between a face outline and 
a background within the head area by specifying a skin color area of the two-dimensional 
image; 

an image processing otcp of creating a person's portrait in which the 
charactcriotico in characteristic features of th e pcroon'o face4s are emphasized based upon 
the two-dimensional image=^y A using data acquired in extracting the head area extraction 
step, the feature detection otcp detecting position of the characteristic features, and=A# 
face outline determining=step a border ; and 

a otop in which, when a communication is received, the person's portrait thus 
created is displayed so as to inform=ti*e a user of=4be receipt of the communication. 
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49. (Amended) A communication method using a communication terminal that 
comprises: 

an image input otcp of p icking up a two-dimensional image containing a person's 
face=%y using an image sensor; 

a head area extracting otcp of extracting a head area from the image picked up=k% 
the imago input atop ; 

a feature detection otcp of detecting position of characteristic features of the face 
for each of the face parts within the head area that has been extracted; 

a face outline determining otcp of determining a border between a face outline and 
a background within the head area; 

an image proooooing atop of creating a person's portrait in which the 
charactoriotico in characteristic features of th e poraon'o face4s are emphasized by 
changing the two-dimensional image with respect to each of the face parts; and 

a atop in which, when a communication is received, the person's portrait thus 
created is displayed so as to inform4h© a user of=4he receipt of the communication. 

50. (Amended) A recording medium in which is recorded a person's portrait 
generation progra m which ia executed for execution by a computer, and compriooo 
comprising : 

a head area extracting program for extracting a head area of a person from a 
differential image of a plurality of two-dimensional images picked up by an image input 
section; 

a feature detection program for detecting positions of characteristic features of the 
person's portrait by dividing the head area into fac e part parts areas for respective face 
parts, finding projection data of binarized images for the respective face parts areas and 
detecting the position of each of the face parts based upon a center-of-gravity position of 
the projection data; 

a face outline determining program for determining a border between a face 
outline and a background within the head area by specifying a skin color area of the two- 
dimensional image; and 
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an image processing program for creating a person's portrait in which the 
characteristics in the person's face4s are emphasized by changing the two-dimensional 
image with respect to each of the face parts. 

Amendments to the abstract: 

ABSTRACT OF THE DISCLOSURE 
¥h#=A person's portrait generation devic e oompriaoo includes an image input 
section which picks up a two-dimensional image containing a person's face^y^ using an 
image sensor. A head area extracting section extracts the head of the person from the 
image. A feature detection section detects the position of characteristic feature(s) of the 
face of that person. A face outline determining section determines a border between a 
face outline and a background. An image processing section generates a person's portrait 
in which the characteristic feature(s) of the face is emphasized. 
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